A Comparison of Transfer Functions for Feature Extracting Layers in the Neocognitron

Many kinds of artiicial neural networks have been applied to the recognition of handwritten characters. Fukushima's neocognitron is one of few networks that demonstrates invariance to input translation and tolerance of a signiicant degree of input distortion and deformation. This paper shows that the behaviour of the neocognitron is dependent upon the form of non-linearity used by the feature extracting components of the network. Preliminary experimental results show that the threshold transfer function performs poorly in comparison to the threshold-linear function used in the original neocognitron. The threshold-linear function is marginally outperformed by the biologically inspired sigmoid function.